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A number of writers have commented on the dearth of substantive research 
within the field of technology education, and point to the expansion of its 
research agenda as a means of strengthening the discipline. Waetjen, in his call 
for good research in technology education, states that “the plea is to use 
experimental type research as much as possible” (1992, p. 30). Interestingly, the 
three areas of research need outlined in his essay would all lend themselves to 
alternative methodologies, including qualitative methodologies. 

More recently, others have called for an expansion in the types of research 
methods used. Of the 220 reports included in Zuga’s review of technology 
education-related research (1994), only 16 are identified as having used 
qualitative methods, and Zuga notes that many of those studies were conducted 
outside the United States. Johnson (1995) suggests that technology educators 
“engage in research that probes for deeper understanding rather than examining 
surface features.” He notes that qualitative methodologies are powerful tools for 
enhancing our understanding of teaching and learning, and that they have 
“gained increasing acceptance in recent years” (p. 4). 

There are compelling reasons for the selection of qualitative methodologies 
within the educational research arena, yet many people remain unfamiliar with 
these methods. Researchers trained in the use of quantitative designs face real 
challenges when called upon to use or teach qualitative research (Stallings, 
1995). There is, however, a growing body of literature devoted to qualitative 
research in education, some of which is synthesized here. The goals of this 
article are to elaborate on the reasons for choosing qualitative methodologies, 
and to provide a basic introduction to the features of this type of research. 

Qualitative Versus Quantitative Research Paradigms 

Researchers have long debated the relative value of qualitative and 
quantitative inquiry (Patton, 1990). Phenomenological inquiry, or qualitative 
research, uses a naturalistic approach that seeks to understand phenomena in 
context-specific settings. Logical positivism, or quantitative research, uses 
experimental methods and quantitative measures to test hypothetical 
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generalizations. Each represents a fundamentally different inquiry paradigm, and 
researcher actions are based on the underlying assumptions of each paradigm. 

Qualitative research, broadly defined, means “any kind of research that 
produces findings not arrived at by means of statistical procedures or other 
means of quantification” (Strauss and Corbin, 1990, p. 17). Where quantitative 
researchers seek causal determination, prediction, and generalization of findings, 
qualitative researchers seek instead illumination, understanding, and 
extrapolation to similar situations. Qualitative analysis results in a different type 
of knowledge than does quantitative inquiry. 

Eisner points out that all knowledge, including that gained through 
quantitative research, is referenced in qualities, and that there are many ways to 
represent our understanding of the world: 

There is a kind of continuum that moves from the fictional that is “true”— 
the novel for example—to the highly controlled and quantitatively 
described scientific experiment. Work at either end of this continuum has 
the capacity to inform significantly. Qualitative research and evaluation 
are located toward the fictive end of the continuum without being fictional 
in the narrow sense of the term (Eisner, 1991, pp. 30-31). 

This sentiment echoes that of an earlier writer. Cronbach (1975) states that “the 
special task of the social scientist in each generation is to pin down the 
contemporary facts. Beyond that, he shares with the humanistic scholar and the 
artist in the effort to gain insight into contemporary relationships” (p. 126). 

Cronbach claims that statistical research is not able to take full account of 
the many interaction effects that take place in social settings. He gives examples 
of several empirical “laws” that do not hold true in actual settings to illustrate 
this point. Cronbach states that “the time has come to exorcise the null 
hypothesis,” because it ignores effects that may be important, but that are not 
statistically significant (1975, p. 124). Qualitative inquiry accepts the complex 
and dynamic quality of the social world. 

However, it is not necessary to pit these two paradigms against one another 
in a competing stance. Patton (1990) advocates a “paradigm of choices” that 
seeks “ methodological appropriateness as the primary criterion for judging 
methodological quality.” This will allow for a “situational responsiveness” that 
strict adherence to one paradigm or another will not (p. 39). Furthermore, some 
researchers believe that qualitative and quantitative research can be effectively 
combined in the same research project (Strauss and Corbin, 1990; Patton, 1990). 
For example, Russek and Weinberg (1993) claim that by using both quantitative 
and qualitative data, their study of technology-based materials for the 
elementary classroom gave insights that neither type of analysis could provide 
alone. 


Basis for the Use of a Qualitative Methodology 
There are several considerations when deciding to adopt a qualitative 
research methodology. Strauss and Corbin (1990) claim that qualitative methods 
can be used to better understand any phenomenon about which little is yet 
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known. They can also be used to gain new perspectives on things about which 
much is already known, or to gain more in-depth information that may be 
difficult to convey quantitatively. Thus, qualitative methods are appropriate in 
situations where one needs to first identify the variables that might later be 
tested quantitatively, or where the researcher has determined that quantitative 
measures cannot adequately describe or interpret a situation. Research problems 
tend to be framed as open-ended questions that will support discovery of new 
information. Greene's 1994 study of women in the trades, for example, asked 
“What personal characteristics do tradeswomen have in common? In what way, 
if any, did role models contribute to women’s choices to work in the trades?” 

(p. 524a). 

The ability of qualitative data to more fully describe a phenomenon is an 
important consideration not only from the researcher’s perspective, but from the 
reader’s perspective as well. “If you want people to understand better than they 
otherwise might, provide them information in the form in which they usually 
experience it” (Lincoln and Guba, 1985, p. 120). Qualitative research reports, 
typically rich with detail and insights into participants’ experiences of the world, 
“may be epistemologically in harmony with the reader’s experience” (Stake, 
1978, p. 5) and thus more meaningful. 

Features of Qualitative Research 

Several writers have identified what they consider to be the prominent 
characteristics of qualitative, or naturalistic, research (see, for example: Bogdan 
and Biklen, 1982; Lincoln and Guba, 1985; Patton, 1990; Eisner, 1991). The list 
that follows represents a synthesis of these authors’ descriptions of qualitative 
research: 

1. Qualitative research uses the natural setting as the source of data. The 
researcher attempts to observe, describe and interpret settings as they 
are, maintaining what Patton calls an “empathic neutrality” (1990, p. 

55). 

2. The researcher acts as the “human instrument” of data collection. 

3. Qualitative researchers predominantly use inductive data analysis. 

4. Qualitative research reports are descriptive, incorporating expressive 
language and the “presence of voice in the text” (Eisner, 1991, p. 36). 

5. Qualitative research has an interpretive character, aimed at discovering 
the meaning events have for the individuals who experience them, and 
the interpretations of those meanings by the researcher. 

6. Qualitative researchers pay attention to the idiosyncratic as well as the 
pervasive, seeking the uniqueness of each case. 

7. Qualitative research has an emergent (as opposed to predetermined) 
design, and researchers focus on this emerging process as well as the 
outcomes or product of the research. 

8. Qualitative research is judged using special criteria for trustworthiness 
(these will be discussed in some detail in a later section). 

Patton (1990) points out that these are not “absolute characteristics of 
qualitative inquiry, but rather strategic ideals that provide a direction and a 
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framework for developing specific designs and concrete data collection tactics” 
(p. 59). These characteristics are considered to be “interconnected” (Patton, 
1990, p. 40) and “mutually reinforcing” (Lincoln and Guba, 1985, p. 39). 

It is important to emphasize the emergent nature of qualitative research 
design. Because the researcher seeks to observe and interpret meanings in 
context, it is neither possible nor appropriate to finalize research strategies 
before data collection has begun (Patton, 1990). Qualitative research proposals 
should, however, specify primary questions to be explored and plans for data 
collection strategies. 

The particular design of a qualitative study depends on the purpose of the 
inquiry, what information will be most useful, and what information will have 
the most credibility. There are no strict criteria for sample size (Patton, 1990). 
“Qualitative studies typically employ multiple forms of evidence.... [and] there 
is no statistical test of significance to determine if results ‘count’” (Eisner, 1991, 
p. 39). Judgments about usefulness and credibility are left to the researcher and 
the reader. 


The Role of the Researcher in Qualitative Inquiry 

Before conducting a qualtitative study, a researcher must do three things. 
First, (s)he must adopt the stance suggested by the characteristics of the 
naturalist paradigm. Second, the researcher must develop the level of skill 
appropriate for a human instrument, or the vehicle through which data will be 
collected and interpreted. Finally, the researcher must prepare a research design 
that utilizes accepted strategies for naturalistic inquiry (Lincoln and Guba, 

1985). 

Glaser and Strauss (1967) and Strauss and Corbin (1990) refer to what they 
call the “theoretical sensitivity” of the researcher. This is a useful concept with 
which to evaluate a researcher’s skill and readiness to attempt a qualitative 
inquiry. 

Theoretical sensitivity refers to a personal quality of the researcher. It 
indicates an awareness of the subtleties of meaning of data. ...[It] refers to 
the attribute of having insight, the ability to give meaning to data, the 
capacity to understand, and capability to separate the pertinent from that 
which isn't (Strauss and Corbin, 1990, p. 42). 

Strauss and Corbin believe that theoretical sensitivity comes from a number of 
sources, including professional literature, professional experiences, and personal 
experiences. The credibility of a qualitative research report relies heavily on the 
confidence readers have in the researcher’s ability to be sensitive to the data and 
to make appropriate decisions in the field (Eisner, 1991; Patton, 1990). 

Lincoln and Guba (1985) identify the characteristics that make humans the 
“instrument of choice” for naturalistic inquiry. Humans are responsive to 
environmental cues, and able to interact with the situation; they have the ability 
to collect information at multiple levels simultaneously; they are able to 
perceive situations holistically; they are able to process data as soon as they 


- 50 - 




Journal of Technology Education 


Vol. 9 No. 1, Fall 1997 


become available; they can provide immediate feedback and request verification 
of data; and they can explore atypical or unexpected responses. 

Research Design and Data Collection Strategies 

Eisner (1991) claims there is a “paucity of methodological prescriptions” 
for qualitative research, because such inquiry places a premium on the strengths 
of the researcher rather than on standardization (p. 169). Lincoln and Guba 
(1985) provide a fairly detailed outline for the design of naturalistic inquiry, 
which includes these general steps: 

1. Determine a focus for the inquiry. This should establish a boundary for 
the study, and provide inclusion/exclusion criteria for new information. 
Boundaries, however, can be altered, and typically are. 

2. Determine the fit of the research paradigm to the research focus. The 
researcher must compare the characteristics of the qualitative paradigm 
with the goals of the research. 

3. Determine where and from whom data will be collected. 

4. Determine what the successive phases of the inquiry will be. Phase one, 
for example, might feature open-ended data collection, while successive 
phases will be more focused. 

5. Determine what additional instrumentation may be used, beyond the 
researcher as the human instrument. 

6. Plan data collection and recording modes. This must include how 
detailed and specific research questions will be, and how faithfully data 
will be reproduced. 

7. Plan which data analysis procedures will be used. 

8. Plan the logistics of data collection, including scheduling and budgeting. 

9. Plan the techniques that will be used to determine trustworthiness. 

Steps one and two have been addressed in previous sections; the remaining steps 
will be addressed below. 

Sampling Strategies for Qualitative Researchers 

In quantitative inquiry, the dominant sampling strategy is probability 
sampling, which depends on the selection of a random and representative sample 
from the larger population. The purpose of probability sampling is subsequent 
generalization of the research findings to the population. By contrast, purposeful 
sampling is the dominant strategy in qualitative research. Purposeful sampling 
seeks information-rich cases which can be studied in depth (Patton, 1990). 

Patton identifies and describes 16 types of purposeful sampling. These 
include: extreme or deviant case sampling; typical case sampling; maximum 
variation sampling; snowball or chain sampling; confirming or disconfirming 
case sampling; politically important case sampling; convenience sampling; and 
others (1990, pp. 169-183). According to Lincoln and Guba (1985), the most 
useful strategy for the naturalistic approach is maximum variation sampling. 

This strategy 
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aims at capturing and describing the central themes or principal outcomes 
that cut across a great deal of participant or program variation. For small 
samples a great deal of heterogeneity can be a problem because individual 
cases are so different from each other. The maximum variation sampling 
strategy turns that apparent weakness into a strength by applying the 
following logic: Any common patterns that emerge from great variation 
are of particular interest and value in capturing the core experiences and 
central, shared aspects or impacts of a program (Patton, 1990, p. 172). 

Maximum variation sampling can yield detailed descriptions of each case, in 
addition to identifying shared patterns that cut across cases. See Hoepfl (1994) 
for an illustration of this strategy applied to technology education research. 
Several examples of studies employing case sampling can also be found in the 
technology education literature (see Brown, 1995; Hansen, 1995; and Lewis, 
1995 and 1997) 

In spite of the apparent flexibility in purposeful sampling, researchers must 
be aware of three types of sampling error that can arise in qualitative research. 
The first relates to distortions caused by insufficient breadth in sampling; the 
second from distortions introduced by changes over time; and the third from 
distortions caused by lack of depth in data collection at each site (Patton, 1990). 

Data Collection Techniques 

The two prevailing forms of data collection associated with qualitative 
inquiry are interviews and observation. 

Interviews 

Qualitative interviews may be used either as the primary strategy for data 
collection, or in conjunction with observation, document analysis, or other 
techniques (Bogdan and Biklen, 1982). Qualitative interviewing utilizes open- 
ended questions that allow for individual variations. Patton (1990) writes about 
three types of qualitative interviewing: 1) informal, conversational interviews; 

2) semi-structured interviews; and 3) standardized, open-ended interviews. 

An interview guide or “schedule” is a list of questions or general topics that 
the interviewer wants to explore during each interview. Although it is prepared 
to insure that basically the same information is obtained from each person, there 
are no predetermined responses, and in semi-structured interviews the 
interviewer is free to probe and explore within these predetermined inquiry 
areas. Interview guides ensure good use of limited interview time; they make 
interviewing multiple subjects more systematic and comprehensive; and they 
help to keep interactions focused. In keeping with the flexible nature of 
qualitative research designs, interview guides can be modified over time to focus 
attention on areas of particular importance, or to exclude questions the 
researcher has found to be unproductive for the goals of the research (Lofland 
and Lofland, 1984). 

Recording Data. A basic decision going into the interview process is how to 
record interview data. Whether one relies on written notes or a tape recorder 
appears to be largely a matter of personal preference. For instance, Patton says 
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that a tape recorder is “indispensable” (1990, p. 348), while Lincoln and Guba 
“do not recommend recording except for unusual reasons” (1985, p. 241). 
Lincoln and Guba base their recommendation on the intrusiveness of recording 
devices and the possibility of technical failure. Recordings have the advantage 
of capturing data more faithfully than hurriedly written notes might, and can 
make it easier for the researcher to focus on the interview. 

Observations 

The classic form of data collection in naturalistic or field research is 
observation of participants in the context of a natural scene. Observational data 
are used for the purpose of description—of settings, activities, people, and the 
meanings of what is observed from the perspective of the participants. 
Observation can lead to deeper understandings than interviews alone, because it 
provides a knowledge of the context in which events occur, and may enable the 
researcher to see things that participants themselves are not aware of, or that 
they are unwilling to discuss (Patton, 1990). A skilled observer is one who is 
trained in the process of monitoring both verbal and nonverbal cues, and in the 
use of concrete, unambiguous, descriptive language. Sours’ (1997) study of 
teaching and learning styles provides a good example of descriptive language 
applied to the technology classroom. 

There are several observation strategies available. In some cases it may be 
possible and desirable for the researcher to watch from outside, without being 
observed. Another option is to maintain a passive presence, being as unobtrusive 
as possible and not interacting with participants. A third strategy is to engage in 
limited interaction, intervening only when further clarification of actions is 
needed. Or the researcher may exercise more active control over the observation, 
as in the case of a formal interview, to elicit specific types of information. 
Finally, the researcher may act as a full participant in the situation, with either a 
hidden or known identity. Each of these strategies has specific advantages, 
disadvantages and concerns which must be carefully examined by the researcher 
(Schatzman and Strauss, 1973). 

The presence of an observer is likely to introduce a distortion of the natural 
scene which the researcher must be aware of, and work to minimize. Critical 
decisions, including the degree to which researcher identity and purposes will be 
revealed to participants, the length of time spent in the field, and specific 
observation techniques used, are wholly dependent on the unique set of 
questions and resources brought to each study. In any case, the researcher must 
consider the legal and ethical responsibilities associated with naturalistic 
observation. 

Recording Data. Field researchers rely most heavily on the use of field 
notes, which are running descriptions of settings, people, activities, and sounds. 
Field notes may include drawings or maps. Acknowledging the difficulty of 
writing extensive field notes during an observation, Lofland and Lofland (1984) 
recommend jotting down notes that will serve as a memory aid when full field 
notes are constructed. This should happen as soon after observation as possible. 
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preferably the same day. In addition to field notes, researchers may use 
photographs, videotapes, and audio tapes as means of accurately capturing a 
setting. 

Gaining Access and Researcher Obligations 

Based on their experience with naturalistic research, Lofland and Lofland 
(1984) believe that researchers are more likely to gain successful access to 
situations if they make use of contacts that can help remove barriers to entrance; 
if they avoid wasting respondents’ time by doing advance research for 
information that is already part of the public record; and if they treat respondents 
with courtesy. Because naturalistic researchers are asking participants to “grant 
access to their lives, their minds, [and] their emotions,” it is also important to 
provide respondents with a straightforward description of the goals of the 
research (p. 25). 

Other Sources of Data 

Another source of information that can be invaluable to qualitative 
researchers is analysis of documents. Such documents might include official 
records, letters, newspaper accounts, diaries, and reports, as well as the 
published data used in a review of literature. In his study of technology teachers 
in training, Hansen (1995) analyzed journal entries and memos written by 
participants, in addition to interviews. Hoepfl (1994), in her study of closure of 
technology teacher education programs, used newspaper reports, university 
policy documents, and department self-evaluation data, where available, to 
supplement data gained through interviews. 

There are some specialized forms of qualitative research which rely solely 
on analysis of documents. For example, Gagel (1997) used a process known as 
hermeneutic inquiry to investigate the literature on both literacy and technology. 
Patton (1990) provides a good overview of the various theoretical orientations 
that inform the “rich menu of alternative possibilities within qualitative 
research” (p. 65). 

Deciding When to Stop Sampling 

Qualitative researchers have few strict guidelines for when to stop the data 
collection process. Criteria include: 1) exhaustion of resources; 2) emergence of 
regularities; and 3) overextension, or going too far beyond the boundaries of the 
research (Guba, 1978). The decision to stop sampling must take into account the 
research goals, the need to achieve depth through triangulation of data sources, 
and the possibility of greater breadth through examination of a variety of 
sampling sites. 


Analysis of Data 

Bogdan and Biklen define qualitative data analysis as “working with data, 
organizing it, breaking it into manageable units, synthesizing it, searching for 
patterns, discovering what is important and what is to be learned, and deciding 
what you will tell others” (1982, p. 145). Qualitative researchers tend to use 
inductive analysis of data, meaning that the critical themes emerge out of the 
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data (Patton, 1990). Qualitative analysis requires some creativity, for the 
challenge is to place the raw data into logical, meaningful categories; to examine 
them in a holistic fashion; and to find a way to communicate this interpretation 
to others. 

Sitting down to organize a pile of raw data can be a daunting task. It can 
involve literally hundreds of pages of interview transcripts, field notes and 
documents. The mechanics of handling large quantities of qualitative data can 
range from physically sorting and storing slips of paper to using one of the 
several computer software programs that have been designed to aid in this task 
(see Brown, 1996, for a description of one of these programs). 

Analysis begins with identification of the themes emerging from the raw 
data, a process sometimes referred to as “open coding” (Strauss and Corbin, 
1990). During open coding, the researcher must identify and tentatively name 
the conceptual categories into which the phenomena observed will be grouped. 
The goal is to create descriptive, multi-dimensional categories which form a 
preliminary framework for analysis. Words, phrases or events that appear to be 
similar can be grouped into the same category. These categories may be 
gradually modified or replaced during the subsequent stages of analysis that 
follow. 

As the raw data are broken down into manageable chunks, the researcher 
must also devise an “audit trail”—that is, a scheme for identifying these data 
chunks according to their speaker and the context. The particular identifiers 
developed may or may not be used in the research report, but speakers are 
typically referred to in a manner that provides a sense of context (see, for 
example. Brown, 1996; Duffee and Aikenhead, 1992; and Sours, 1997). 
Quantitative research reports are characterized by the use of “voice” in the text; 
that is, participant quotes that illustrate the themes being described. 

The next stage of analysis involves re-examination of the categories 
identified to determine how they are linked, a complex process sometimes called 
“axial coding” (Strauss and Corbin, 1990). The discrete categories identified in 
open coding are compared and combined in new ways as the researcher begins 
to assemble the “big picture.” The purpose of coding is to not only describe but, 
more importantly, to acquire new understanding of a phenomenon of interest. 
Therefore, causal events contributing to the phenomenon; descriptive details of 
the phenomenon itself; and the ramifications of the phenomenon under study 
must all be identified and explored. During axial coding the researcher is 
responsible for building a conceptual model and for determining whether 
sufficient data exists to support that interpretation. 

Finally, the researcher must translate the conceptual model into the story 
line that will be read by others. Ideally, the research report will be a rich, tightly 
woven account that “closely approximates the reality it represents” (Strauss and 
Corbin, 1990, p. 57). Many of the concerns surrounding the presentation of 
qualitative research reports are discussed in the section “Judging Qualitative 
Research” which follows. 

Although the stages of analysis are described here in a linear fashion, in 
practice they may occur simultaneously and repeatedly. During axial coding the 
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researcher may determine that the initial categories identified must be revised, 
leading to re-examination of the raw data. Additional data collection may occur 
at any point if the researcher uncovers gaps in the data. In fact, informal analysis 
begins with data collection, and can and should guide subsequent data 
collection. For a more detailed yet very understandable description of the 
analysis process, see Simpson and Tuson (1995). 

The Product of Qualitative Data Analysis 

In their classic text Discovery of Grounded Theory, Glaser and Strauss 
(1967) describe what they believe to be the primary goal of qualitative research: 
the generation of theory, rather than theory testing or mere description. 
According to this view, theory is not a “perfected product” but an “ever- 
developing entity” or process (p. 32). Glaser and Strauss claim that one of the 
requisite properties of grounded theory is that it be “sufficiently general to be 
applicable to a multitude of diverse situations within the substantive area” (p. 
237). 

The grounded theory approach described by Glaser and Strauss represents a 
somewhat extreme form of naturalistic inquiry. It is not necessary to insist that 
the product of qualitative inquiry be a theory that will apply to a “multitude of 
diverse situations.” Examples of a more flexible approach to qualitative inquiry 
can be gained from a number of sources. For example, both Patton (1990) and 
Guba (1978) state, in the same words, that “naturalistic inquiry is always a 
matter of degree” of the extent to which the researcher influences responses and 
imposes categories on the data. The more “pure” the naturalistic inquiry, the less 
reduction of data into categories. 

Figure 1 illustrates one interpretation of the relationship between 
description, verification, and generation of theory—or, in this case, the 
development of what Cronbach (1975) calls “working hypotheses,” which 
suggests a more tractable form of analysis than the word “theory.” According to 
this interpretation, a researcher may move between points on the description/ 
verification continuum during analysis, but the final product will fall on one 
particular point, depending on the degree to which it is naturalistic. 


Naturalistic/Inductive Logico/Deductive 

Inquirylnquiry 


' - WORKING HYPOTHESES -► 

DESCRIPTION VERIFICATION 

Figure 1. Description, verification and generation of working hypotheses in 
qualitative research. 

In keeping with a naturalistic stance, the researcher might conclude that, to 
the extent that findings are based on information from a variety of diverse 
situations, they may be applicable to a larger substantive area. However, their 
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applicability to a particular situation is wholly dependent upon the conditions of 
the situation and the usefulness of the research findings to individual readers. 

Judging Qualitative Research 

The Role of the Reader 

Those who are in a position to judge or use the findings of a qualitative 
inquiry must play a different type of role than people who review quantitative 
research. This is because “there are no operationally defined truth tests to apply 
to qualitative research” (Eisner, 1991, p. 53). Instead, researcher and readers 
“share a joint responsibility” for establishing the value of the qualitative 
research product (Glaser and Strauss, 1967, p. 232). “Pragmatic validation [of 
qualitative research] means that the perspective presented is judged by its 
relevance to and use by those to whom it is presented: their perspective and 
actions joined to the [researcher’s] perspective and actions” (Patton, 1990, p. 
485). 

Eisner (1991) believes that the following three features of qualitative 
research should be considered by reviewers: 

Cohei'ence: Does the story make sense? How have the conclusions been 
supported? To what extent have multiple data sources been used to give 
credence to the interpretation that has been made? (p. 53). 

Related to coherence is the notion of “structural corroboration,” also known as 
triangulation (p. 55). 

Consensus: The condition in which the readers of a work concur that the 
findings and/or interpretations reported by the investigator are consistent 
with their own experience or with the evidence presented (p. 56). 

Finally, reviewers must assess the report’s: 

Instrumental Utility. The most important test of any qualitative study is its 
usefulness. A good qualitative study can help us understand a situation 
that would otherwise be enigmatic or confusing (p. 58). 

A good study can help us anticipate the future, not in the predictive sense of the 
word, but as a kind of road map or guide. “Guides call our attention to aspects of 
the situation or place we might otherwise miss” (Eisner, 1991, p. 59). 

Addressing Trustworthiness in Qualitative Research 
The basic question addressed by the notion of trustworthiness, according to 
Lincoln and Guba, is simple: “How can an inquirer persuade his or her 
audiences that the research findings of an inquiry are worth paying attention to?” 
(1985, p. 290). When judging qualitative work, Strauss and Corbin (1990) 
believe that the “usual canons of ‘good science’.. .require redefinition in order to 
fit the realities of qualitative research” (p. 250). Lincoln and Guba (1985, 
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p. 300) have identified one alternative set of criteria that correspond to those 
typically employed to judge quantitative work ( see Table 1). 


Table 1 

Comparison of criteria for judging the quality of quantitative versus qualitative 
research 


Conventional terms 

Naturalistic terms 

internal validity 

credibility 

external validity 

transferability 

reliability 

dependability 

objectivity 

confirmability 


Smith and Heshusius (1986) sharply criticize those writers, like Lincoln and 
Guba, who they believe have adopted a stance of “detente” with rationalists. 
They are particularly incensed by Lincoln and Guba’s use of “comparable 
criteria,” which to their eyes look little different than the conventional criteria 
they supposedly replace. In either case, there must be a “belief in the assumption 
that what is known—be it an existent reality or an interpreted reality—stands 
independent of the inquirer and can be described without distortion by the 
inquirer” (p. 6). Smith and Heshusius claim that naturalistic research can offer 
only an “interpretation of the interpretations of others,” and that to assume an 
independent reality is “unacceptable” for the qualitative researcher (p. 9). 

Their stance is a strong one, because the only reality it accepts is a 
completely mind-dependent one, which will vary from individual to individual; 
in other words, for Smith and Heshusius, there is no “out there” out there. For 
these researchers, it would not be possible to choose a best interpretation from 
among the many available, because no technique or interpretation can be 
“epistemologically privileged” (p. 9). To maintain this stance would seem to 
negate the value of doing research at all, because it prohibits the possibility of 
reconciling alternative interpretations. 

Therefore, it is important to determine which criteria are consistent with the 
naturalistic paradigm, yet which allow for a declaration that “good science” has 
been carried out. In the following sections, conventional and naturalistic criteria 
will be discussed, with the goal of selecting criteria which are appropriate for 
judging the overall trustworthiness of a qualitative study. 

Internal Validity versus Credibility 

In conventional inquiry, internal validity refers to the extent to which the 
findings accurately describe reality. Lincoln and Guba (1985) state that “the 
determination of such isomorphism is in principle impossible” (p. 294), because 
one would have to know the “precise nature of that reality” and, if one knew this 
already, there would be no need to test it (p. 295). The conventional researcher 
must postulate relationships and then test them; the postulate cannot be proved, 
but only falsified. The naturalistic researcher, on the other hand, assumes the 
presence of multiple realities and attempts to represent these multiple realities 
adequately. Credibility becomes the test for this. 
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Credibility depends less on sample size than on the richness of the 
information gathered and on the analytical abilities of the researcher (Patton, 
1990). It can be enhanced through triangulation of data. Patton identifies four 
types of triangulation: 1) methods triangulation; 2) data triangulation; 3) 
triangulation through multiple analysts; and 4) theory triangulation. Other 
techniques for addressing credibility include making segments of the raw data 
available for others to analyze, and the use of “member checks,” in which 
respondents are asked to corroborate findings (Lincoln and Guba, 1985, pp. 313- 
316). 

External Validity / Generalizability versus Transferability 

In conventional research, external validity refers to the ability to generalize 
findings across different settings. Making generalizations involves a trade-off 
between internal and external validity (Lincoln and Guba, 1985). That is, in 
order to make generalizable statements that apply to many contexts, one can 
include only limited aspects of each local context. 

Lincoln and Guba (1985) admit that generalizability is “an appealing 
concept,” because it allows a semblance of prediction and control over situations 
(pp. 110-111). Yet they suggest that the existence of local conditions “makes it 
impossible to generalize” (p. 124). Cronbach (1975) discusses the problem by 
saying: 

The trouble, as I see it, is that we cannot store up generalizations and 
constructs for ultimate assembly into a network. It is as if we needed a 
gross of dry cells to power an engine and could only make one a month. 

The energy would leak out of the first cells before we had half the battery 
completed (p. 123). 

According to Cronbach, “when we give proper weight to local conditions, any 
generalization is a working hypothesis, not a conclusion” (p. 125). 

In the naturalistic paradigm, the transferability of a working hypothesis to 
other situations depends on the degree of similarity between the original 
situation and the situation to which it is transferred. The researcher cannot 
specify the transferability of findings; he or she can only provide sufficient 
information that can then be used by the reader to determine whether the 
findings are applicable to the new situation (Lincoln and Guba, 1985). Other 
writers use similar language to describe transferability, if not the word itself. For 
example. Stake (1978) refers to what he calls “naturalistic generalization” (p. 6). 
Patton suggests that “extrapolation” is an appropriate term for this process 
(1990, p. 489). Eisner says it is a form of “retrospective generalization” that can 
allow us to understand our past (and future) experiences in a new way (1991, 
p. 205). 

Reliability versus Dependability 

Kirk and Miller (1986) identify three types of reliability referred to in 
conventional research, which relate to: 1) the degree to which a measurement, 
given repeatedly, remains the same; 2) the stability of a measurement over time; 
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and 3) the similarity of measurements within a given time period (pp. 41-42). 
They note that “issues of reliability have received little attention” from 
qualitative researchers, who have instead focused on achieving greater validity 
in their work (p. 42). Although they give several examples of how reliability 
might be viewed in qualitative work, the essence of these examples can be 
summed up in the following statement by Lincoln and Guba (1985): “Since 
there can be no validity without reliability (and thus no credibility without 
dependability), a demonstration of the former is sufficient to establish the latter” 
(p. 316). 

Nevertheless, Lincoln and Guba do propose one measure which might 
enhance the dependability of qualitative research. That is the use of an “inquiry 
audit,” in which reviewers examine both the process and the product of the 
research for consistency (1985, p. 317). 

Objectivity versus Confirmability 

Conventional wisdom says that research which relies on quantitative 
measures to define a situation is relatively value-free, and therefore objective. 
Qualitative research, which relies on interpretations and is admittedly value- 
bound, is considered to be subjective. In the world of conventional research, 
subjectivity leads to results that are both unreliable and invalid. There are many 
researchers, however, who call into question the true objectivity of statistical 
measures and, indeed, the possibility of ever attaining pme objectivity at all 
(Lincoln and Guba, 1985; Eisner, 1991). 

Patton (1990) believes that the terms objectivity and subjectivity have 
become “ideological ammunition in the paradigms debate.” He prefers to “avoid 
using either word and to stay out of futile debates about subjectivity versus 
objectivity.” Instead, he strives for “empathic neutrality” (p. 55). While 
admitting that these two words appear to be contradictory, Patton points out that 
empathy “is a stance toward the people one encounters, while neutrality is a 
stance toward the findings” (p. 58). A researcher who is neutral tries to be non- 
judgmental, and strives to report what is found in a balanced way. 

Lincoln and Guba (1985) choose to speak of the “confirmability” of the 
research. In a sense, they refer to the degree to which the researcher can 
demonstrate the neutrality of the research interpretations, through a 
“confirmability audit.” This means providing an audit trail consisting of 1) raw 
data; 2) analysis notes; 3) reconstruction and synthesis products; 4) process 
notes; 5) personal notes; and 6) preliminary developmental information (pp. 320 
-321). 

With regard to objectivity in qualitative research, it may be useful to turn to 
Phillips (1990), who questions whether there is really much difference between 
quantitative and qualitative research: 

Bad work of either kind is equally to be deplored; and good work of either 
kind is still—at best—only tentative. But the good work in both cases will 
be objective, in the sense that it has been opened up to criticism, and the 
reasons and evidence offered in both cases will have withstood serious 
scrutiny. The works will have faced potential refutation, and insofar as 
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they have survived, they will be regarded as worthy of further 
investigation (p. 35). 


Discussion and Conclusion 

The increased interest in qualitative research in recent years warrants a 
basic understanding of this paradigm on the part of all technology education 
researchers. This overview of qualitative research methods and issues represents 
a starting point only for those who are interested in using and/or reviewing 
qualitative research. Readers can choose from a growing body of literature on 
the topic for further guidance. 

The decision to use qualitative methodologies should be considered 
carefully; by its very nature, qualitative research can be emotionally taxing and 
extraordinarily time consuming. At the same time, it can yield rich information 
not obtainable through statistical sampling techniques. 

In the past, graduate students contemplating the use of qualitative inquiry 
were told that they would have to “sell” the idea to members of their research 
committees, who would probably view qualitative research as inferior to 
quantitative research. Fortunately, in most universities that belief has changed, 
to the point where qualitative research is the paradigm of choice in some 
schools. In spite of this growing acceptance, new researchers may still encounter 
difficulties in finding faculty advisors who are skilled in this type of research. 

Qualitative researchers have a special responsibility to their subjects and 
their readers. Since there are no statistical tests for significance in qualitative 
studies, the researcher bears the burden of discovering and interpreting the 
importance of what is observed, and of establishing a plausible connection 
between what is observed and the conclusions drawn in the research report. To 
do all of this skillfully requires a solid understanding of the research paradigm 
and, ideally, guided practice in the use of qualitative observation and analysis 
techniques. 

There are many useful research designs, the selection of which depends on 
the research questions being asked. Most importantly, technology educators 
must rise to the challenge to find and use rigorous, appropriate research 
techniques that address the significant questions facing the field. 
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